One aspect of hardware and software testing involves discovering “regressions” or slow operation of computer hardware or software. In the modern era of automated testing, thousands of tests may be run prior to releasing a product on the market. Regressions discovered in this rigorous testing process should be investigated, which typically requires human investigation into the cause of regressions.
The modern computing system is a complex environment. To simulate real-world conditions, it is desirable to test in this complex environment. For example, a software test may run while various unrelated or tangentially related hardware or software components are initialized on a computing device. Such additional hardware or software components may impact the operation of software under test. In fact, it is desirable to test under a number of different hardware and software configurations to simulate the variety of complex environments under which hardware or software components may function.
In a field where a large number of tests may be run in complex environments, and problems require human investigation, it is important to ensure that identified regressions are in fact caused by the computer hardware or software under test. Time spent investigating regressions that are caused by “noise” or some uninteresting aspect of the complex environment is often time wasted.
In view of the need in the industry to improve efficiency in software testing, there is a need to account for noise in hardware and software testing.